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ABSTRACT 

The paper presents a comprehensive approach to 
evaluation of programs for post secondary students with learning 
disabilities. The evaluation design is suitable for programs in 
2-year and 4-year settings, and incorporates both process and product 
evaluation methods. The approach was used to evaluate learning 
disability support services at Housatonic Community College 
(Connecticut) and the University of Connecticut. The evaluation 
focused on program activities, such as identifying referral sources, 
processing referrals, implementing direct instructional services, 
providing testing accommodations, and consulting with on-campus 
personnel; and program outcomes, such as satisfactory completion of 
courses, retention, and graduation. Examples are offered of forms 
generated to manage ongoing data collection and to display 
longitudinal information regarding program activities and outcomes. 
Future areas for evaluation of service delivery are outlined. 
(JDD) 
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ABSTRACT 

As services for students with learning disabilities continue to 
develop in postsecondary settings, it is important to implement 
evaluation activities, not only to demonstrate program effective- 
ness but also to identify areas for improvement to better meet 
student needs* This paper presents a comprehensive approach 
used in Connecticut to develop an evaluation design suitable for 
programs in two- and four-year settings. Through cooperative 
efforts of personnel from the Department of Higher Education, 
program directors and an independent evaluator, a model 
incorporating process and product evaluation methods was 
applied This design may have relevance for other institutions 
considering similar evaluation needs* 
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Given the fact that students with learning disabilities represent the fastest growing category 
of students with disabilities seeking access to higher education, learning disability college 
programs are multiplying rapidly to address their needs. Yet efficacy data pertaining to program 
models are sparse, which, in a period of limited fiscal resources, places serious restrictions on 
budget requests. Over a two-year period (1984-1986), the Connecticut State Legislature 
funded two model programs for college students with learning disabilities. To demonstrate 
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the efficacy of these programs, joint efforts of the Department of Higher Education, an . 
independent evaluator, and program directors resulted in a comprehensive model of program 
evaluation which can be replicated and adapted in other settings. 

The purposes of this article are as follows: (a) to describe the involvement of the 
Connecticut State Legislature and Department of Higher Education in sponsoring two pilot 
programs for students with learning disabilities; (b) to discuss the evaluation model and 
process used to demonstrate the effectiveness of these programs; and (c) to identify future 
areas for evaluation of service delivery to such students. 



Program Development Background 

In Connecticut, concern about the lack of support services for students with learning 
disabilities at the postsecondary level had been expressed by students, parents, and advocacy 
groups who lobbied at the state level for program funding* These efforts resulted in passage of 
a special legislative act in May, 1984, which authorized and funded over a two-year period two 
pilot programs, one at a two-year and one at a fourryear public institution. The Learning 
Disability Advisory Committee was established by the Department of Higher Education* to 
provide expertise and to select the recipients of the grant Criteria for reviewing proposals 
submitted to the Department were determined, and applicants woe required to meet with the 
Committee as well as to document institutional commitment to continue support services 
following the legislative funding period. 

A requirement of the funding act was an evaluative study of the pilot programs to be . 
conducted by an independent evaluator who met in the fall of 1984 with project directors at 
Housatonic Community College and the University of Connecticut, sites of the support 
programs. Legislative funding in the initial year for both programs and evaluation activities 
totaled $55,000. 



Evaluation Activities and Results 

Overall, goals of the pilot programs were outlined in the legislative act and included 
diagnostic evaluation, individualized instruction, consultation with faculty, and testing 
accommodations. Within the context of these service areas, an evaluation plan was outlined, 
discussed with project directors, and implemented at each institution during the first semester 
of program operations. 

In planning the evaluation process and procedures, the Context-Input-Process-Product 
(CIPP) model (Stufflebeam, Foley, Gephart, Cuba, Hammond, Memman, & Provus, 1972) 
was adapted. Stufflebeam, et al., define educational evaluation as "the process of delineating, 
.obtaining, and providing useful information forjudging decision alternatives" (p. 40). Figure 1 
illustrates the emphasis placed on process and product evaluation activities since context and 
input variables were addressed by each institution as part of the grant proposal review 
procedure. Both schools had demonstrated a critical need for support services for students with 
learning disabilities. Additional on-campus resources such as Counseling Services, Disabled 
Student" Services, Math and Writing Centers, and Mental Health Services were identified as 
ancillary supports which could be utilized by the students. Personnel requirements, budget 
allocations, and facilities and space needs were factors included in the grant applications. 

So that efficient data collection techniques c** #1 d be systematically applied, the independent 
evaluator met with each program director to dL , methods for obtaining information and to 
establish a timeline for collecting data. Two major aspects of the programs constituted the focus 
for evaluation: 
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a. Program process, or the manner in which the program implemented activities designed to 
meet program goals. 43 

b. Program product, or the results of providing services designed to assist students with 
learning disabilities. 



Context 



What is the context 
in which the program 
will operate? 

What needs underlie 
program development? 

What resources 
already exist? 



Input 



Ptocess 



Product 



What program services 
are needed to meet 
program goals? 

What are th* personnel 
requirements for program 
implementation? 

What are the budgeting 
considerations? 



What methods and 
activities are imple- 
mented to operation- 
tionalize program 
goals and objectives? 



What are the outcomes 
achieved through program 
implementation? 



What are space and 
facilities needs? 



Adapted from Stufflebeam, D;L, Foley, W. J. f Gephard, W. J. t Guha, E. G., Hammond, H. a, Merriman, 
H. O., and Provus, M. M. (1972). Educational evaluation and decision-making. Itasca, EL: Peacock. 

Figure !• Evaluation questions adapted from the CDPP model 



Given the scope of the evaluation component as well as the audience for the evaluation 
report, several measurement techniques were used to generate descriptive statistics and 
anecdotal evidence regarding program implementation and outcomes. Figure 2 provides an 
overview of areas for evaluation and data collection methods. 

In conducting evaluation activities, it is important to develop forms which provide a clear 
and concise overview of data. Program directors and the evaluator generated a number of tables 
which were useful in managing on-going data collection and displaying longitudinal informa- 
tion regarding program activities and outcomes. Several examples of these forms are illustrated 
in Figure 3. 

Interim reports and briefings regarding the evaluation process were important in 
substantiating to the Legislature that funds were being used to facilitate educational opportu- 
nities at the postsecondary level for students with learning disabilities. In both settings, mean 
pde point averages for students with learning disabilities receiving support services were 
higher than the required 2.0 for satisfactory academic status. Retention rates of 82% at 
Housatonic Community College and 92% at the University of Connecticut demonstrate 
without doubt that postsecondary education is a realistic and attainable goal for qualified 
students with learning disabilities. Data from the evaluation study were used effectively to 
lobby for additional legislative funding. In 1986, the Legislature approved funds for an 
additional program at another state four-year institution and for two consortia which provide 
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consultation and on-site technical assistance to public and independent institutions interested in 
implementing support programs. 

Additionally, as Stufflebeam, et al., suggest, evaluation should be a dynamic process 
through which "activities are evaluated to influence decisions, which influence activities, 
which are in turn evaluated, ad infinitum" (p. 215). As a result of the evaluation process, both 
programs have revised or modified various facets of service delivery based upon analyses of 
data. It is this process of using evaluation findings for decision-making which underlies 
Stufflebeam s statement that "the purpose of evaluation is not to prove but to improve." 



Focus of Evaluation Data Collection Methods 



Review of: 
Project logs ' • 

Project logs^:-; " r : v 
Staff reports and weekly summaries 
Project logs and staff reports 
Administrative calendar and log 



Review of: 

Transcripts and dean's list 
Grade point averages by semester 
Transcripts and commencement lists •". \ 



Figure 2. Areas for evaluation and methods of gathering data 



Table 2 

Summary of Referral Follow-up Activities 



Follow-up Activity 


1984/85 


1985/86 


Intake interview 






Psychoeducational evaluation 






Referred to other sources 






LD program services 







Figure 3. Example of data collection forms 



Program Activities 

Identifying referral sources 
Processing referrals 

Implementing direct instructional services 
Providing testing accommodations 
Consulting with on-campus personnel 



Program Outcomes 

Satisfactory completion of courses 
Retention 
Graduation 
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Tabled 

Summary of Direct Services 



ot Mudents Served 


Mean Hours per Week 


1984/85 1985/86 


1984/85 1985/86 


Individualized instruction 




Group instruction 




Consultation 




Table 5 




Semester Summary of Grade Point Averages 




Semester 




Cnrinrr 108< Coll 1GOC 

opnng iyoD rail lyoj 


Spnng 1986 


Number of students 




Mean GPA unweighted 




Mean GPA weighted 




Range 




Mean credit hours 





Figure 3. continued 
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Table 6 

Overview of Student Status 



Semester 



Fall 1986 Spring 1987 

Lower division 
Upper division 
Graduated 
Dismissed 
Withdrew 
Dean's list 



Figure 3. continued 



Future Directions 

The commitment to provide equal educational opportunities at the postsecondary level for 
students with learning disabilities is obvious. As we in AHSSPPE look ahead to the second- 
decade of promoting equality for individuals with disabilities, we again are in a leadership 
position to demonstrate the benefits of implementing Section 504. Accountability should 
serve as a catalyst to objectively analyze services offered to students so that commitment to 
continually improve is grounded in sound data-based information. Many issues relating to 
college students with learning disabilities need to be systematically investigated: accessibility 
as it is affected by admissions policies; longitudinal studies to determine outcomes of service 
delivery at the postsecondary level; efficacy of instructional approaches and compensatory 

Slick. tC^lC S • 

The approach and model described in this paper yielded results which were influential in 
funding issues at state and institutional levels. Efforts to refine evaluation procedures and 
interface with other campus resources using such data are underway in both settings. As a 
larger data base emerges, use of inferential statistical techniques will provide additional 
findings forjudging decision alternatives and ultimately improving educational opportunities 
for students with learning disabilities. 
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